Dynamic File-access Characteristics of a Production Parallel Scientiic Workload

Multiprocessors have permitted astounding increases in computational performance, but many cannot meet the intense I/O requirements of some scientiic applications. An important component of any solution to this I/O bottleneck is a parallel le system that can provide high-bandwidth access to tremendous amounts of data in parallel to hundreds or thousands of processors. Most successful systems are based on a solid understanding of the expected workload, but thus far there have been no comprehensive workload characterizations of multiprocessor le systems. This paper presents the results of a three week tracing study in which all le-related activity on a massively parallel computer was recorded. Our instrumentation diiers from previous eeorts in that it collects information about every I/O request and about the mix of jobs running in a production environment. We also present the results of a trace-driven caching simulation and recommendations for designers of multiprocessor le systems.